Method for transmitting data using an embedded bit stream produced in a hierarchical table-lookup vector quantizer

ABSTRACT

A transmission method for video image data using an embedded bit stream in a hierarchical table-lookup vector quantizer comprises the steps encoding an image using hierarchical vector quantization and an embedding process to obtain an embedded bit stream for lossless transmission. The bit stream is selectively truncated and decoded to obtain a reconstructed image.

BACKGROUND OF THE INVENTION

[0001] This invention relates to a method for producing an embedded bitstream in a hierarchical table lookup vector quantizer for use inconnection with image data compression wherein an image is encoded at atransmitter, transmitted, and/or selectively decoded at a receiver. Anembedded bit stream is one in which any prefix of the bit stream is avalid bit stream of the image at a lower rate. The present inventionprovides transcoding to bit streams of arbitrary lower rate, simply bytruncation. Thus, image data can be transmitted in an embedded bitstream and the receiver can use as much of the embedded bit stream as isnecessary to reconstruct the image to a desired or allowable resolution.That is, the invention implements progressive decoding of bit streams toimages with increasing resolution as bits arrive at the receiver.

[0002] While the invention is particularly directed to the art of imagedata compression, and will thus be described with specific referencethereto, it will be appreciated that the invention has applicability toother media, such as video and audio. Exemplary applications to suchmedia include packet priorization for bandwidth scalability in networks,information prioritization for unequal error protection for wirelesscommunication, and arbitrarily fine bit rate control at the encoder.

[0003] By way of background, image and video coding standards such asJPEG, MPEG, and MPEG-II are based on transform coding. In transformcoding, a block transform such as the discrete cosine transform (DCT) istypically applied to 8×8 blocks of an image, and each transformcoefficient or frequency b d is independently quantized. Thequantization stepsizes typically vary from band to band, with largerstepsizes in the higher frequency bands reflecting the fact thatquantization errors at higher spatial frequencies are visually lessapparent.

[0004] JPEG has a progressive transmission mode, in which an image isstored or transmitted with the most significant bits first, then thenext most significant bits, and so on. This is accomplished by beginningwith large stepsizes for each band, and successively halving thestepsizes in selected bands in some sequence. As each stepsize ishalved, additional bits are stored or transmitted. The resulting bitstream is said to be embedded in the sense that any prefix of the bitstream can be decompressed into an image whose quality is appropriatefor the length of the prefix. That is, lower resolution encodings areembedded in higher resolution encodings. Such an embedded bit stream isuseful, for example, in image retrieval or telebrowsing applications inwhich the user must scan through a large collection of images to find adesired image, and it is too expensive or time-consuming to reproduceall the images in full detail. In these applications, the decoder canprogressively reconstruct images of improving quality as bits arrive.

[0005] Most research on image and video coding algorithms still focuseson variants of transform coding. In particular, the transform is oftenbased on wavelet analysis, subband filtering, or lapped orthogonaltransforms, for which the transform blocks overlap, thereby reducingobjectionable blocking artifacts.

[0006] J. M. Shapiro, “Embedded Image Coding Using Zerotrees of WaveletCoefficients,” IEEE Trans. on Signal Processing, Vol. 41, No. 17, pp.3445-3463, December 1993, and D. Taubman and A. Zakhor, “Multi-Rate 3-DSubband Coding of Video,” IEEE Trans. Image Proc., Vol. 3, No. 5, pp.572-588, September 1994, employ wavelet transform coding in aprogressive mode. Taubman uses the resulting embedded bit stream forsimple bit rate control in a video encoder. Bit rate control isessential when coupling a Ad variable-rate data compressor to afixed-rate channel. If the channel can transmit exactly (or a maximumof) B bits per second, then the bit rate of the compressor must bedecreased or increased when the buffer between the compressor andchannel begins to overflow or underflow, respectively. With an embeddedbit stream, bit rate control becomes as simple as taking a prefix of anappropriate length from the encoding for each frame (or other unit) ofthe video data.

[0007] Vector quantization is a generalization of simple (scalar)quantization, in which each vector of coefficients is simultaneouslyquantized using a small number of bits, e.g. 8, to represent the vectoras the index of one of, for example, 256 possible reproduction vectors(called codewords) in a collection of reproduction vectors (called thecodebook). A special case of vector quantization is scalar quantizationapplied independently to each component of a vector. The codewords inthis case are constrained to lie on a rectangular lattice. In vectorquantization, this constraint is removed. Hence vector quantization issuperior to scalar quantization in that it can arrange the codewords inthe vector space for maximum coding efficiency. For example, thecodewords can be arranged to populate only the probable regions of thespace. This can be accomplished using a training set of typical data anda clustering algorithm. See A. Gersho and R. M. Gray, “VectorQuantization and Signal Compression,” Kluwer Academic Publishers, 1992.

[0008] Not only do vector quantizers have superior rate-distortionperformance; they also permit decoding by simple table lookup. When thedecoder receives an 8 bit code, for example, it reproduces the originalvector by using the code as the index to a table, and reading thereproduction vector out of the table. If all the coefficients of a blocktransform are blocked into a single vector, the decoder can decode thewhole block with a simple table lookup. No inverse transform isnecessary in this case, as it would be if the components of thetransform were scalar quantized.

[0009] Unfortunately, there are some drawbacks to vector quantization.The first is encoder complexity. Since the codebook is unstructured, foreach input vector, the encoder must perform a full search through thecodebook looking for the codeword that would result in the lowestdistortion. This is often excessively time-consuming for practicalalgorithms. A second related problem is that for a given bit rate (inbits per component), the number of bits per vector grows linearly withvector dimension, and hence the number of codewords grows exponentiallywith vector dimension (and hence so does encoder complexity, and encoderand decoder memory requirements). As a practical consequence, the vectordimension must match the bit rate. At low bit rates, large vectordimensions are feasible. At higher bit rates, only small dimensions maybe feasible. At the highest bit rates, only scalar quantization may befeasible. At any given bit rate, the best rate-distortion performancewill be attained with the largest feasible vector dimensions.

[0010] To reduce the computational complexity of searching the codebookof a vector quantizer, a number of alternatives have been developed. Oneof these is tree-structured vector quantization. In tree-structuredvector quantization, the codewords are arranged in a tree structure(typically a binary tree), with one codeword at each node. Instead ofsearching the codebook exhaustively, it can be searched in a timelogarithmic fashion by beginning at the root node, successivelycomparing the input vector to the codewords at each branch node,descending to the branch with the lower distortion, and repeating theprocess until reaching a leaf node. The codeword at the leaf is thedesired reproduction vector and is represented by a binary code for theleaf, 8 bits to specify one of 256 leaves, for example. If the tree iscomplete, that is, if all leaves are at the same depth, (e.g. 8), thenthe path map to the leaf may be used as its index.

[0011] The path map has the embedded code property: any prefixrepresents a reproduction vector in a coarser codebook, i.e., at ahigher level in the tree. Trees whose leaves are at varying depthsresult in a variable-rate code, in which the binary strings used torepresent the reproduction vector vary in length depending on the inputvector. Since fixed-rate codes are a special case of variable-ratecodes, variable-rate codes can always do at least as well as fixed-ratecodes (typically 20-30% better), at the cost of extra delay due tobuffering, and reduced tolerance to bit errors.

[0012] The embedded coding property applies to variable-ratetree-structured vector quantizers as well as fixed-rate quantifers. E.A. Riskin, T. Lookabaugh, P. A. Chou, and R. M. Gray, “Variable-RateVector Quantization for Medical Image Compression,” IEEE Trans. MedicalImaging, Vol. 9, No. 3, pp. 290-298, September 1990; W. -J. Hwang and H.Derin, “Multi-Resolution Multi-Rate Progressive Image Transmission,”Proc. 27th Asilomar Conf. on Signals, Systems, and Computers, Nov. 1-3,1993. Pacific Grove, Calif., pp. 65-69; and, M. Effros, P. A. Chou, E.A. Riskin, and R. M. Gray, “A Progressive Universal Noiseless Coder,”IEEE Trans. Information Theory, Vol. 40, No. 1, pp. 108-117, January1994, explore variable-rate tree-structured vector quanitization forprogressive image transmission. In the latter work, the tree-structuredvector quantizer is structured so that the codewords lie on arectangular lattice. As a result, the complete tree represents the inputvector losslessly. Thus the progressive transmission sequence results ina lossless coding.

[0013] Another method developed for reducing the full search encodingcomplexity of unstructured vector quantization is hierarchicaltable-lookup vector quanitization, developed by P.- C. Chang, J. May,and R. M. Gray, “Hierarchical Vector Quantization with Table-LookupEncoders,” Proc. Int'l. Conf. on Communications, Chicago, Ill., June1985, pp. 1452-55. In hierarchical table-lookup vector quantization, ahierarchy of moderately sized tables (e.g., 64 Kbytes) is used toperform the encoding. (Decoding is performed as usual in vectorquantization, by a single table lookup.) For example, with referenceFIG. 1a, to encode an 8-dimensional input vector into 8 bits, eachcomponent is first finely quantized to 8 bits, or one byte (this isusually already the case in image coding). Then each of the 4 adjacentpairs of bytes is used to address a 16-bit table (Table 1), to produceone byte for each pair. This process is repeated in two additionallevels of the hierarchy, resulting in a single byte for the8-dimensional input vector.

[0014] Each Table 1, 2 and 3 is 64 Kbytes. The number of tables is thelogarithm of the vector dimension. The computational complexity of thescheme is at most one table lookup per input symbol, since thecomplexity of the first level is ½ table lookup per input symbol, thecomplexity of the second level is ¼ table lookup per input symbol, andso on. Each table lookup implements, approximately, the equivalent of anencoder full search unstructured vector quantizer. Table 1 encodes eachpossible 2-dimensional vector (or rather, each of 64K possible pairs ofbytes) to the index of the best or lowest distortion codeword in a2-dimensional codebook of the size 256. Table 2 encodes each possible4-dimensional vector (or rather, each of the 64K possible 4-dimensionalvectors reproducible after the first level of quantization) to the indexof the best or lowest distortion codeword in a 4-dimensional codebook ofthe size 256, and so on for each level.

[0015] A signal flow diagram for such an HVQ encoder is shown in FIG.1b. In the HVQ of FIG. 1b, the tables T at each stage of the encoderalong with the delays Z are illustrated. Each level in the hierarchydoubles the vector dimension of the quantizer, and therefore reduces thebit rate by a factor of 2. By similar reasoning, the ith level in thehierarchy performs one lookup per 2′ samples, and therefore the totalnumber of lookups per sample is at most ½+¼+⅛+. . .=1, regardless of thenumber of levels. Of course, it is possible to vary these calculationsby adjusting the dimensions of the various tables.

[0016] The contents of the HVQ tables can be determined in a variety ofways. A straightforward way is the following. With reference to FIG. 1a,Table 1 is simple a table-lookup version of an optimal 2-dimensional VQ.That is, an optimal 2-dimensional full search vector quantizer with M=256 codewords is designed by standard means (e.g., the generalizedLloyd algorithm discussed by A. Gersho and R. M. Gray, “VectorQuantization and Signal Compression,” Kluwer Academic Publishers, 1992),and Table 1 is filled so that it assigns to each of its 216 possible2-dimensional input vectors the 8-bit index of the nearest codeword.

[0017] Table 2 is just slightly more complicated. First, an optimal4-dimensional full search VQ with M=256 codewords is designed bystandard means. Then Table 2 is filled so that it assigns to each of its216 possible 4-dimensional input vectors (i.e., the cross product of allpossible 2-dimensional output vectors from the first stage) the 8-bitindex of its nearest codeword. The tables for stages 3 and up aredesigned similarly. Note that the distortion measure is completelyarbitrary.

[0018] This same structure may be used for vector quantization of thecoefficients in a block transform (such as a discrete cosine transform,DCT). In this case, the transform computation may be embodied in thetable lookups, as studied by N. Chaddha, M. Vishwanath, and P. A. Chou,“Hierarchical Vector Quantization of Perceptually Weighted BlockTransforms,” Proc. Data Compression Conference, Snowbird, VT, April1995. A sliding window version of the structure may be used for vectorquantization of the coefficients of wavelet, subband, or lappedtransforms, as studied by M. Vishwananth and P. A. Chou, “An EfficientAlgorithm for Hierarchical Compression of Video,” Proc. Int'l. Conf. onImage Processing, Austin, Tex., Nov. 1994, Vol. 3, pp. 275-279. In thiscase as well, the transform computation is embodied in the tables. It isalso a simple matter to embody arbitrary perceptual distortion measuresin the tables.

[0019] Thus, hierarchical vector quantization offers both extremely lowcomputational complexity and good rate-distortion performance. It alsooffers a simple means of transcoding (reformatting) the bit stream atrate R bits per second to another bit stream at rate R/2 (or R/4, R/8,etc.) bits per second by further table-lookups to map each pair of bytesinto a single byte. In M. Vishwananth and P. A. Chou, “An EfficientAlgorithm for Hierarchical Compression of Video,” Proc. Int'l. Conf. onImage Processing, Austin, Tex., November 1994, Vol. 3, pp. 275-279, itwas argued that such simple transcoding would be useful for reducing thebit rate of the compressed signal at gateways between high and lowcapacity networks.

[0020] Unfortunately, transcoding by table lookup is not simple enoughfor certain applications. It is undesirable to placeapplication-dependent algorithms (such as table look-ups for videocoding) at network gateways or switches. Embedded coding is a simplerway, not dependent on the application, for the gateway to transcode fromhigh rate to low rate streams. Equivalently, receivers on the low ratenetwork can subscribe to only the high priority streams. This is calledbandwidth scalability. If an embedded bit stream is packaged intoprioritized bit streams, then the gateway need only pass on to the lowrate network the high priority streams. No application-specificprocessing is done at the gateway. A gateway node need only be able tothreshold packet numbers, which is a capability that will most likely bepresent in future network protocols.

[0021] Another use of embedded coding, or prioritized bit streams, in apacket network is congestion control. As packet buffers overflow, lowpriority packets can be dropped, and the signal reconstructed from theremaining packets will be gracefully degraded. A similar idea applies towireless communications. If packets of information are prioritized, thenit is possible to use unequal error protection on the different packets.That is, the highest priority packets will be channel coded for maximumerror protection, and the lowest priority packets may not have any errorcorrection applied, before modulation and transmission. The encoder neednot have any particular knowledge of packet loss or channelcharacteristics. It need only rank bits in terms of priority.

[0022] A final use of embedded coding, particularly for fixed-ratenetworks, is for rate control, as mentioned above. Many of these uses ofembedded coding are simultaneously applicable.

[0023] The present invention contemplates a new and improved image datatransmission method which resolves the shortcomings of prior schemes.

SUMMARY OF THE INVENTION

[0024] A method for encoding and decoding an image is provided. Themethod, which is implemented in a complementary apparatus, comprisesreceiving an image and performing an encoding process involvinghierarchical table lookup vector quantization on blocks of the image andembedding resultant data to obtain an embedded bit stream of datarepresenting the image. Any selected prefix of the bit stream representsa valid bit stream of the image at a lower rate than the entire bitstream.

[0025] In accordance with another aspect of the present invention, theembedded bit stream is transmitted to a receiver.

[0026] In accordance with another aspect of the present invention, thetransmitted bit stream may be truncated during transmission.

[0027] In accordance with another aspect of the invention, a decodingoperation is performed on the truncated embedded bit stream to obtain areconstructed image.

[0028] In accordance with another aspect of the invention, decodingoperations are successively performed on the embedded bit stream toobtain progressively improved reconstructed images as the bits arrive atthe decoder.

[0029] One advantage of the present invention is that an improved methodof transcoding from high rate to low rate bit streams is provided.

[0030] Another advantage of the present invention is that an entirevector representing an image can be losslessly encoded such that everyprefix of the string is a valid code for the entire vector so that areceiver can us as much of the embedded bit stream as is necessary toreconstruct the image to a desired or allowable resolution.

[0031] Another advantage of the present invention is that this losslessencoding/decoding is accomplished using only table lookups.

[0032] Further scope of the applicability of the present invention willbecome apparent from the detailed description provided below. It shouldbe understood, however, that the detailed description and specificexamples, while indicating preferred embodiments of the invention, aregiven by way of illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art.

DESCRIPTION OF THE DRAWINGS

[0033] The present invention exists in the construction, arrangementand/or combination, of the various parts and steps, whereby the objectscontemplated are attained and hereinafter more fully set forth,specifically pointed out in the claims, and illustrated in theaccompanying drawings in which:

[0034]FIGS. 1a and 1 b illustrate a known approach for hierarchicaltable-lookup vector quantization;

[0035]FIG. 2 shows a representation of an embedded bit stream;

[0036]FIG. 3 is a flowchart generally representing the method accordingto the present invention;

[0037]FIGS. 4a and 4 b collectively represent the structure of anextended tree-structured codebook at levels 1-5 according to the presentinvention;

[0038]FIGS. 5a and 5 b collectively represent the structure of anextended tree-structured codebook at level 6 according to the presentinvention;

[0039]FIG. 6 represents a binary tree that extends from each of thecodewords of FIGS. 4b and 5 b;

[0040]FIG. 7 shows a data structure for encoding according to thepresent invention;

[0041]FIG. 8 is a flowchart illustrating the encoding procedure of thepresent invention; and,

[0042]FIG. 9 is a flowchart illustrating the decoding procedure of thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] A method of producing an embedded bit stream in a hierarchicaltable lookup vector quantizer is provided. The embedded bit stream hasthe property that at the highest rate, the reproduction of theunderlying data is lossless. However, because the bits are arranged inorder of importance and embedded such that any prefix of the bit streamis a valid bit stream of lower rate, the bit stream may be truncated atany point and a complete image reconstructed. FIG. 2 shows arepresentation of an embedded bit stream B having a most significant bitMSB suitably positioned. The bits decrease in importance in the streamalong the direction identified by the arrow D.

[0044] A simple way to produce an embedded bit stream using hierarchicaltable lookup vector quantization is to replace the full search codebookembodied in the last table in the hierarchy by a tree-structuredcodebook. Then the index output from the table will be an embedded code.Thus the final table could output 16 bits per vector, for example, andthe decoder could reproduce a codeword for any prefix of the 16 bitstring. This approach to embedding in hierarchical table-lookup vectorquantization is analogous to the approach to embedding in interbandtransform VQ taken by N. Chaddha, P. A. Chou, and T. H.-Y. Meng,“Scalable Compression based on Tree Structured Vector Quantization ofPerceptually Weighted Block, Lapped, and Wavelet Transforms,” Proc. IEEEInt'l Conf. on Image Processing, Oct. 23-26, Washington, DC, 1995, pp.89-92.

[0045] While this approach is viable, it is limited in its utility,because the string size is limited to approximately 16 bits. Sixteen(16) bits per vector is a rather crude level of quantization. Somehigher speed networks, for example, may only need to compress the datato 128 bits per vector. Lower capacity networks may tolerate only 4 bitsper vector.

[0046] The present invention provides for encoding of an entire vectorlosslessly with a long string, such that every prefix of the string is avalid code. Furthermore, this is accomplished using only table lookups.

[0047] Accordingly, a long embedded code that losslessly codes data isproduced using hierarchical table lookup vector quantization, asfollows. For simplicity of discussion, it is assumed that each table isaddressed with two 8-bit indices. A codebook at all stages, includingthe final stage, of a hierarchical table lookup vector quantizer isreplaced with a tree-structured vector quantizer with the property thatthe leaves of the tree are in 1-1 correspondence with the entries in thecorresponding table (of which there are typically 64 K). Such a tree canbe designed using a variety of methods, discussed briefly below.

[0048] Because a tree-structured vector quantizer is used, any prefix ofa path to a leaf represents a codeword. When the entire path to a leafis transmitted to the decoder, then because the leaf corresponds toexactly one entry in the corresponding table, the inputs to the tableare known exactly at the decoder. These inputs represent the outputs ofthe tables at the previous stage. Assuming that these tables also embodytree-structured vector quantizers whose leaves are in 1-1 correspondencewith its entries, the output from each of these tables is the first 8bits in the path map from the root to a leaf. Additional bits providethe remainder of the path map, and hence identify the leaf, and itscorresponding entry, and hence identify the inputs to the table at thatlevel. This procedure can be repeated all the way back up the hierarchyof table lookups to the input signal, which can be identified losslesslywith sufficient numbers of bits. In the case that every tree-structuredvector quantizer is fixed- rate (i.e., the trees are complete, with allits leaves at the same depth, in this case 16), then the number of bitsin the longest embedded string is the number of bits in the inputvector, in this case 8 times the vector dimension. For a vectordimension of 8×8 (typical in image coding), this is a string of length8×8×8=512 bits. Any prefix of this string can be decoded to a level ofquality appropriate for the length of the prefix.

[0049] A feature of the invention is that each table implements amany-to-one mapping (the quantizer) such that when supplemented withadditional information, the mapping becomes invertible. This enables thedecoder to work backwards up the hierarchy if it is given the additionalinformation. It is important that the many-to-one mapping (thequantizer), by itself, represents the data with low distortion;although, of course, it cannot represent the data with zero distortion,since it is a many-to-one mapping. Thus one possibility is to design afixed or variable rate tree-structured vector quantizer as usual forthis mapping. This will ensure that the many-to-one mapping representsthe data with as low a distortion as possible given that there can beonly 256 possible outputs, for example.

[0050] Now, each leaf of the tree represents an output of the mapping,and therefore corresponds to the equivalence class of table entries withthe same output. Each equivalence class has a certain size. For eachequivalence class, a fixed or variable rate tree structure can bedesigned such that each leaf corresponds to an element of the class.Taken together, these tree structures define a one-to-one mappingbetween table entries (elements) and binary strings from an embeddedcode.

[0051] In a typical application to bandwidth scalability in networks, asgenerally illustrated in FIG. 3, an 8-bit (monochrome) image is obtained(step 10) and encoded into an embedded bit stream that represents a pathmap (step 12). The bit stream is then transmitted in multiple streams ofvarying priority within the network, the low priority streams beingsubject to (step 14) being pruned away or truncated so that the higherpriority streams can be transmitted over a lower bandwidth channel (step16). The higher priority bit streams are then decoded into thereconstructed image (step 18).

[0052] Other applications—which may be implemented as an alternative orin addition to bandwidth scalability as described above—are alsoillustrated in FIG. 3. Steps 24, 26 and 28 illustrate unequal errorprotection. That is, after the image is encoded, different levels oferror protection coding to bits of different significance are provided(step 24) and the protected bit-stream is transmitted (step 26).Transmission errors are then corrected and the bit stream is truncatedat an uncorrectable error (step 28). The truncated bit stream is thendecoded to obtain a reconstructed image (step 18).

[0053] Rate control at the encoder is represented in FIG. 3. Moreparticularly, after the image is encoded at step 12, the resultantembedded bit stream is truncated to a desired rate (step 20) and thetruncated bit stream is transmitted (step 22). The truncated bit streamis then decoded and the reconstructed image is obtained (step 18).

[0054] Progressive transmission is also shown. After the image isencoded, the resultant embedded bit stream is transmitted (step 30). Thetransmitted embedded bit stream is then decoded progressively as bitsarrive at the receiver to obtain the reconstructed image (step 32).

[0055] As those skilled in the art will appreciate, the encoding anddecoding processes involve the use of codebooks and corresponding tableswhich are constructed based on the codebooks. Identical sets ofcodebooks are respectively available at both the transmitting andreceiving ends of the transmission of the embedded bit stream (path map)while the corresponding tables are only used at the transmitting end inthe preferred embodiment. It is to be further appreciated that thememory chips utilized to store the codebooks and tables are preferablyread only memory (ROM's). However, it is recognized that other suitablememory means such as PROM, EPROM, EEPROM, RAM, etc. may be utilized.

[0056] As those skilled in the art will further appreciate, the methodas described herein may be implemented in any suitable hardwareconfiguration and/or by any suitable control scheme. In the preferredembodiment, the hardware configuration comprises a general purposecomputer adapted to facilitate the present invention. For example,suitable tables are configured in memory and buffers are established.The preferred control scheme is software based and includes instructionswritten in the C language. It is contemplated that various controlprocedures in various mediums could be used.

[0057] The details of how 1) the codebooks and tables are constructed,2) the image is encoded into an embedded bit stream, and 3) thetruncated bit stream is decoded into the reconstructed image, will nowbe more particularly described.

Construction of Codebooks and Tables

[0058] For purposes of explanation, the image will be coded in 8×8blocks, so that six levels of codebooks and tables will be required,1=1,2,3,4,5,6, corresponding to vector dimensions 2×1, 2×2, 4×2, 4×4,8×4, and 8×8, respectively. For the first five of these levels,codebooks and tables are designed in the usual, known way forhierarchical table-lookup vector quantization, as described for examplein P. -C. Chang, J. May, and R. M. Gray, “Hierarchical VectorQuantization with Table-Lookup Encoders,” Proc. Int'l. Conf. onCommunications, Chicago, Il., June 1985, pp. 1452-55. That is to say,the codebook at level 1 (1 1,2,3,4,5) is a regular full-search vectorquantization codebook with 256 2^(L)-dimensional codewords, sayc_(±)(i), i=0, . . . ,255. A representation of a codebook for levels 1-5is shown in FIG. 4a. The table at level 1(1=1,2,3,4,5) contains in its(j,k)th entry the index i of the codeword c_(±)(i) to which the 2

-dimensional vector (c_(±−1)(j),c_(±−1)(k)) is quantized by thefull-search vector quantizer at level 1.

[0059] For the last level 1 (1=6), the codebook and table are designeddifferently. The codebook is designed as a binary tree-structuredcodebook with depth 8, as described for example in A. Gersho and R. M.Gray, “Vector Quantization and Signal Compression,” Kluwer AcademicPublishers, 1992, and shown in FIG. 5a. The table is constructed tocontain in its (j,k)th entry the index i (actually, the 8-bit binarypath map) of the codeword c_(±)(i) to which the 2

-dimensional vector (c_(±−1)(j),c_(±−1)(k)) is quantized by thetree-structured vector quantizer.

[0060] As illustrated in FIGS. 4b and 5 b, according to the presentinvention and beyond the above described processes, the codebook at eachlevel 1 (1=1,2,3,4,5,6) is extended into a variable-lengthtree-structured vector quantizer by growing a binary tree from eachcodeword c_(±)(i), i=0, . . . , 255, in the codebook at that level. Thisis accomplished by quantizing a training sequence with the codebook atlevel 1-1, pairing the quantized 2

⁻¹-dimensional vectors into 2

-dimensional vectors, and then using the resulting sequence as atraining set to grow the quantizer at level 1 into a variable-lengthtree-structured vector quantizer with the maximum number of leaves, asshown in FIGS. 4b and 5 b. Since there are only 64K possible

2

-dimensional vectors in the quantized training sequence, there will beat most 64K leaves of the maximal tree. The leaves of the binary subtreeemanating from codeword c_(±)(i) correspond to those indices (j,k) thatmap to index i at level 1.

[0061] Each node of the maximal tree is labeled with its codeword, asusual. In addition, as shown in FIG. 6—which illustrates an exemplarybinary tree emanating from codewords c_(±)(i) of FIGS. 4b and 5 b—eachleaf of the tree is labeled with its corresponding (j,k) pair, and eachinternal node of the tree is labeled with a “refinement index”—r, r+1,r+2, r+3, r+4. The refinement index is an integer that indicates theperceptual significance of the bit in the path map that emanates fromthe node. Refinement indices range from 0 to N−1, where N is the numberof refinements desired in the embedded bit stream. Bits with refinementindex 0 are most significant, while bits with refinement index N−1 areleast significant. Thus the refinement indices are non-decreasing alongpaths from the root of a tree to its leaves. Furthermore, the refinementindices in the tree at level 1-1 will tend to be higher than therefinement indices in the tree at level 1. In fact, the refinementindices of the nodes near the root of the tree at level 6 will be 0,while the refinement indices of the nodes near the leaves of the tree atlevel 1 will be N−1. Bits are placed into the embedded bit stream inorder of their refinement indices.

[0062] Although a wide variety of ways could be used, one way to computethe refinement indices is as follows. A set of N+1 Lagrange multipliersλ₀, λ₁, . . . ,λ_(N), is chosen such that ∞=λ₀<λ₁, <λ₂<. . .<λ_(N)=0.Then the tree is pruned at each level to each Lagrange multiplier. (Thepruning procedure is the generalized BFOS algorithm and can be found forexample in A. Gersho and R. M. Gray, “Vector Quantization and SignalCompression,” Kluwer Academic Publishers, 1992.) For the tree at anygiven level, this produces a set of nested pruned subtrees, sayS₀≦S₁≦S₂≦. . . ≦S_(N) such that S₀ contains only the root node and SN isthe maximal tree. A node is then labeled by refinement index r if thenode is an internal node of S_(±+1), but not an internal node of S_(±).In addition, each of these nested subtrees S₀, . . . , . . . S_(r+1). .. S_(N−1), S_(N) corresponds to the multipliers λ₀, . . . λ_(r),λ_(r+1), . . .λ_(N−1), λ_(N).

[0063] Just as the codebooks are extended, the tables are extended. The(j,k)th entry of the table at level 1 contains the 8-bit index i of thecodeword c+(i) in the codebook at level 1. The (j,k)th entry in thetable is extended to include the path map b₁b₂b₃ . . . b

from the node containing c_(_(l)i) to the leaf labeled (j,k) . Thelength len of this path map varies with (j,k). Also included in the(j,k)th entry is the sequence of refinement indices r₁r₂r₃. . . r

along the path map. The extended table at level 1 is substantially ofthe form as follows: (0, 0) len b₁b₂b₃ . . . b_(ten) r₁r₂r₃ . . .r_(ten) (0, 1) (0, 2) . . . . . . . . . . . . . . . (255, 255)

Encoding

[0064] The encoding operation comprises performing hierarchical tablelookup vector quantization wherein a fixed tree data structure isutilized for at least two quantization levels and embedding resultantdata by selecting bits from at least two levels of the fixed tree datastructure to obtain a first bit stream having a first rate representingthe image. A selected prefix of the first bit stream comprises a secondbit stream representing the image at a second rate.

[0065] More particularly, to encode an image, N buffers are established,labeled 0 through N−1. The rth buffer will contain all the bits withrefinement index r.

[0066] Next, each 8×8 block of the image is evaluated, in turn. For each8×8 block, the block is encoded using ordinary HVQ, down to 6 levels.The HVQ process is generally described in connection with FIGS. 1a and 1b. As the encoding proceeds, however, the fixed data structure shown inFIG. 7 is filled in.

[0067] As shown, this data structure is a tree structure and has 32nodes at level 1, 16 nodes at level 2, and so forth, down to a singlenode at level 6. The nth node at level 1 corresponds to the nth subblockof dimension 2₌ ¹ within the 8×8 block. At some time during the ordinaryHVQ encoding process, this subblock will be encoded into an index i bylooking up the (j,k)th entry in the table at level 1. At that time, theinformation (len, b₁b₂b₃. . . b

, r₁r₂r₃. . . r

) from the (j,k)th entry in the extended table at level 1 should becopied into the nth node at level 1 in the data structure above.

[0068] Once the nodes in the data structure are completely filled in,the bits {b _(m) } in this data structure are put into the N buffersaccording to their associated refinement indices {r_(m} as follows. For refinement index r going from) 0 to N−1, thefollowing recursive procedure is performed, beginning with the root nodeof the structure: embed(rootnode,r).

[0069] The definition of the embed procedure is the following:

[0070] embed(node,r) {

[0071] for m=1 to node.len do

[0072] if node.b[m] is marked as already output then continue

[0073] else if node.r[m]<=r then

[0074] output node.b[m] to buffer r

[0075] mark node.b[m] as having been output

[0076] else if node.r[m]>r then return

[0077] if node is not at level 1, then

[0078] embed(leftsubnode,r)

[0079] embed(rightsubnode,r) }

[0080] This procedure outputs all bits in the node up through refinementindex r, if they haven't been output already. When all the bits in thenode have been exhausted, then the procedure is called on the left andright subnodes.

[0081] When the refinement index reaches N, then the bits at allrefinement levels have been output for this block. The remaining blocksare processed in a similar way. When all the blocks have been processed,the buffers containing the bits at the various refinement levels may beconcatenated together into one long embedded bit stream, or they may bestored in individual files, transmitted through individual channels withdifferent levels of error protection, etc.

[0082] A flow chart of the encoding procedure is shown in FIG. 8.Initially, buffers 0 through N−1 are emptied (step 100) and an 8×8 blockof the image to be transmitted is selected (step 102). Hierarchialvector quantization of the block, filling in the data structure of FIG.7, is performed (step 104). Refinement index r is next set to zero (0)(step 106). The recursive embedding procedure described above isperformed (step 108) and a determination is made whether r=N (step 110).

[0083] If r≠N, then r is incremented (step 112) and the embeddingprocedure (step 108) and (step 110) are again performed. If r=N, adetermination is made whether the 8×8 block encoded is the last block ofthe total image (step 114). If the 8×8 block is not the last block, thenthe next 8×8 block is selected (step 116) and steps 104-114 areperformed. If the 8×8 block is the last block of the image, then thecontents of buffers 0 through N−1 are output (step 118). The contents ofthese buffers thus represent the embedded bit stream, or path map, thatis transmitted to a receiver, truncated, and decoded to reconstruct theencoded image.

Decoding

[0084] The decoding operation comprises recovering a selected number ofbuffers, decoding contents of the recovered buffers by following pathmaps through the fixed tree data structure to leaves of the treestructure at which are stored portions of the image to reconstruct theimage, and outputting the reconstructed image.

[0085] To that end, the codebooks can be used for decoding an image. Forsimplicity, it will be assumed that the buffers for the differentrefinement levels (0 through N−1) have been stored in individual files,or have been transmitted through individual channels, so that they canbe recovered before any decoding is done. This allows the image to bedecoded in a single pass through its 8×8 blocks. If the buffers havebeen concatenated into one long embedded bit stream, then the image mustbe decoded in multiple passes. Each pass reads more bits from theembedded bit stream and progressively enhances the image.

[0086] Referring now to FIG. 9, the truncation, decoding andreconstruction are illustrated in the flowchart therein. First, anarbitrary number R is selected according to the available bandwidth forthe receiver (step 200). The last of these N buffers may thus betruncated. This is equivalent to taking an arbitrary prefix of theembedded bit stream. The first R of the original N buffers, labeled 0through R−1, are then recovered (step 202).

[0087] A first 8×8 block of the image is selected (step 204) and decodedaccording to the following procedure, starting with the root node of thecodebook at level 6, and r=R:

[0088] decode(node,r,level) {

[0089] while node is not a leaf do

[0090] if node.r (the node's refinement index)>r then

[0091] output node.c (the node's codeword) return

[0092] if buffer node.r is empty, return

[0093] read a bit out of buffer node.r

[0094] if the bit is zero then

[0095] node=left subnode (i.e., go left)

[0096] else node=right subnode (i.e., go right)

[0097] We're at a leaf. Get its labels (j,k)

[0098] if level=1 then

[0099] output pixel values j,k

[0100] return

[0101] decode(rootnode of jth subtree at level l−1,r,l-1)

[0102] decode(rootnode of kth subtree at level l−1,r,l−1) }

[0103] A determination is then made as to whether the last block of theimage has been decoded (step 208). If the last block has not beendecoded, the next 8×8 block is selected (step 210) and steps 206 and 208are repeated. If the last block has been decoded, the reconstructedimage is output (step 212).

[0104] Thus, an image can be encoded, transmitted, and reconstructuredto a variety of different resolutions. This variety of resolutionsincludes that resolution representing a lossless transmission.

[0105] The above description only provides a disclosure of theparticular embodiments of the invention and is not intended for thepurpose of limiting the same thereto. As such, the invention is notlimited only to the above described embodiments. Rather, it isrecognized that one skilled in the art could conceive alternativeembodiments that fall within the scope of the invention.

Having thus described the invention, we now claim:
 1. A transmissionmethod for image data, the method comprising the steps of: receiving animage; performing an encoding operation on the image comprisingperforming hierarchical table lookup vector quantization wherein a fixedtree data structure is utilized for at least two quantization levels andembedding resultant data by selecting bits from at least two levels ofthe fixed tree data structure to obtain a first bit stream having afirst rate representing the image, a selected prefix of the first bitstream comprising a second bit stream representing the image at a secondrate; is transmitting one of the first bit stream and the second bitstream; and, successively performing a decoding operation on one of thefirst bit stream and the second bit stream to obtain a reconstructedimage.
 2. The method as set forth in claim 1 further comprising storingthe first bit stream in buffers during the encoding.
 3. The method asset forth in claim 2 wherein the decoding operation comprises recoveringa selected number of the buffers, decoding contents of the recoveredbuffers by following path maps through the fixed tree data structure toleaves of the tree structure at which are stored portions of the imageto reconstruct the image, and outputting the reconstructed image.
 4. Themethod as set forth in claim 1 wherein the first bit stream is alossless bit stream.
 5. The method as set forth in claim 1 wherein thesecond bit stream comprises the first bit stream.
 6. A transmissionmethod for image data, the method comprising the steps of: receiving animage; performing an encoding operation on the image comprisingperforming hierarchical table lookup vector quantization wherein a fixedtree data structure is utilized for at least two quantization levels andembedding resultant data by selecting bits from at least two levels ofthe fixed tree data structure to obtain a first bit stream having afirst rate representing the image, a selected prefix of the first bitstream comprising a second bit stream representing the image at a secondrate; performing at least one of 1) truncating the first bit stream toobtain the second bit stream and transmitting the second bit stream, 2)transmitting the first bit stream in a manner representing a pluralityof bit streams of varying priority and truncating low priority bitstreams of the plurality to obtain the second bit stream, 3) providingdifferent levels of error protection coding to bits of differingsignificance in the first bit stream, transmitting the first bit streamwith the error protection coding, attempting to correct transmissionerrors in the first bit stream, and truncating the first bit stream atan uncorrectable error based on the error protection coding to obtainthe second bit stream, and, 4) transmitting the first bit stream; and,successively performing a decoding operation on one of the first bitstream and the second bit stream by following path maps through thefixed tree data structure to leaves of the tree structure at which arestored portions of the image to obtain a reconstructed image.
 7. Themethod as set forth in claim 6 further comprising storing the first bitstream in buffers during the encoding.
 8. The method as set forth inclaim 7 wherein the decoding operation comprises recovering a selectednumber of the buffers, decoding contents of the recovered buffers, andoutputting the reconstructed image.
 9. The method as set forth in claim6 wherein the first bit stream is losslessly encoded.
 10. A methodcomprising: constructing a first codebook in accord with hierarchicaltable lookup vector quantization having a predetermined number of firstcodewords to be accessed by a full search of the first codebook;constructing a second codebook that is provided with a first treestructure with a predetermined depth having a predetermined number ofsecond codewords accessed based on a binary path map based on the firsttree structure; growing second tree structures from each first andsecond codeword; constructing tables based on the first and secondcodebooks; obtaining an image; performing an encoding operation on theimage comprising performing hierarchical table lookup vectorquantization wherein the tables are accessed and a third tree structureto store data is utilized for at least two quantization levels andembedding resultant data by selecting bits from at least two levels ofthe fixed tree data structure to obtain a first bit stream having afirst rate representing the image, a selected prefix of the first bitstream comprising a second bit stream representing the image at a secondrate; and, performing at least one of 1) truncating the first bit streamto obtain the second bit stream and transmitting the second bit stream,2) transmitting the first bit stream in a manner representing aplurality of bit streams of varying priority and truncating low prioritybit streams of the plurality to obtain the second bit stream, 3)providing different levels of error protection coding to bits ofdiffering significance in the first bit stream, transmitting the firstbit stream with the error protection coding, attempting to correcttransmission errors in the first bit stream, and truncating the firstbit stream at an uncorrectable error based on the error protectioncoding to obtain the second bit stream, and, 4) transmitting the firstbit stream.
 11. The method as set forth in claim 10 further comprising:successively performing a decoding operation on one of the first bitstream and the second bit stream by following path maps through thefixed tree data structure to leaves of the tree structure at which arestored portions of the image to obtain a reconstructed image.
 12. Anapparatus comprising: means for receiving an image; means for performingan encoding operation on the image comprising performing hierarchicaltable lookup vector quantization wherein a fixed tree data structure isutilized for at least two quantization levels and embedding resultantdata by selecting bits from at least two levels of the fixed tree datastructure to obtain a first bit stream having a first rate representingthe image, a selected prefix of the first bit stream comprising a secondbit stream representing the image at a second rate; means for performingat least one of 1) truncating the first bit stream to obtain the secondbit stream and transmitting the second bit stream, 2) transmitting thefirst stream in a manner representing a plurality of bit streams ofvarying priority and truncating low priority bit streams of theplurality of obtain the second truncated bit stream, 3) providingdifferent levels of error protection coding to bits of differingsignificance in the first bit stream, transmitting the first bit streamwith the error protection coding, attempting to correct transmissionerrors in the first bit stream, and truncating the first bit stream atan uncorrectable error based on the error protection coding to obtainthe second bit stream, and, 4) transmitting the first bit stream; and,means for performing a decoding operation on one of the first bit streamand the second bit stream by following path maps through the fixed treedata structure to leaves of the tree structure at which are storedportions of the image to obtain a reconstructed image.
 13. The apparatusas set forth in claim 12 further comprising buffers for storing thefirst bit stream during the encoding.
 14. The apparatus as set forth inclaim 13 wherein the means for decoding comprises means for recovering aselected number of the buffers based on the second bit stream, means fordecoding contents of the recovered buffers, and means for outputting thereconstructed image.
 15. A method comprising: constructing a firstcodebook in accord with hierarchical table lookup vector quantizationhaving a predetermined number of first codewords to be accessed by afull search of the first codebook; constructing a second codebook thatis provided with a first tree structure with a predetermined depthhaving a predetermined number of second codewords accessed based on abinary path map based on the first tree structure; growing second treestructures from each first and second codeword; constructing tablesbased on the first and second codebooks; obtaining an image; performingan encoding operation on the image comprising performing hierarchicaltable lookup vector quantization wherein the tables are accessed and athird tree structure to store data is utilized for at least twoquantization levels and embedding resultant data by selecting bits fromat least two levels of the fixed tree data structure to obtain a firstbit stream having a first rate representing the image, a selected prefixof the first bit stream comprising a second bit stream representing theimage at a second rate; transmitting one of the first bit stream and thesecond bit stream; and, successively performing a decoding operation onone of the first bit stream and the second bit stream by following pathmaps through the fixed tree data structure to leaves of the treestructure at which are stored portions of the image to obtain areconstructed image.